Systems and methods for modifying a user interface based on eye focus

ABSTRACT

Disclosed is a computing system and associated methods that use changes in eye focus or the depth at which a user is looking to modify a user interface. The computing system presents a three-dimensional (“3D”) environment with different user interface (“UI”) elements that are positioned in a foreground or near plane of the 3D environment and that partially or wholly obscure a background or far plane of the 3D environment. The computing system detects a change in user eye focus from the foreground to the background by using a sensor to track changes to the pupil or the amount of light reflecting off the user&#39;s eye. The computing system produces an unobstructed view to all or part of the background by adjusting positioning, opacity, or other properties of the UI elements in the foreground in response to detecting the change in the user eye focus.

BACKGROUND

Virtual reality, augmented reality, mixed reality, spatial computing,and/or other three-dimensional (“3D”) environments present windows,objects, and/or other user interface (“UI”) elements at different planesor depths. UI elements in the 3D environment foreground may restrict theview of the 3D environment background. For instance, the 3D environmentbackground may include other UI elements or a view of a real-world scenethat is wholly or partially obscured by the UI elements in the 3Denvironment foreground.

Hand gestures, 3D controllers, and keyboards may be used to reveal the3D environment background. For instance, the user provideshand-generated inputs to change what is presented in the 3D environmentor the layering of UI elements in the 3D environment. The hand gestures,3D controller inputs, and/or keyboard inputs may not be intuitiveactions for changing what is presented in the 3D environment and/or forswitching between the 3D environment foreground and background.Specifically, the hand gestures, 3D controllers, and keyboards requirediscrete inputs that are generated with the user's hands to change thepresentation of the 3D environments or the UI elements presentedtherein.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an example of performing a user interface (“UI”)action based on eye focus tracking in accordance with some embodimentspresented herein.

FIG. 2 illustrates example components of a device for performing the UIadjustments based on tracked changes to the user's eye focus inaccordance with some embodiments presented herein.

FIG. 3 illustrates eye tracking examples performed by an eye focustracking sensor in accordance with some embodiments presented herein.

FIG. 4 presents a process for controlling a UI using eye focus inaccordance with some embodiments.

FIG. 5 illustrates examples of the different adjustments for exposingthe 3D environment background or far plane in accordance with someembodiments presented herein.

FIG. 6 illustrates an example of different UI adjustments that areimplemented based on controls derived from different combinations of eyefocus and eye position tracking in accordance with some embodimentspresented herein.

FIG. 7 illustrates an example of adapting the eye focus controls forswitching between UI elements in accordance with some embodimentspresented herein.

FIG. 8 illustrates example components of one or more devices, accordingto one or more embodiments described herein.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

The following detailed description refers to the accompanying drawings.The same reference numbers in different drawings may identify the sameor similar elements.

Provided are systems and methods for modifying a user interface (“UI”)based on eye focus. The systems and methods track a user's eye focus orthe depth at which the user is looking, and automatically perform one ormore UI actions based on the eye focus. Specifically, the UI actions areimplemented without requiring separate hand-generated input from handgestures, three-dimensional (“3D”) controllers, keyboards, and/or otherinput devices operated with the user's hand. The one or more UI actionsinclude adjusting the positioning or transparency of UI elements in theforeground in order to reveal the UI elements or the scene at differentdepths or planes in the background.

Tracking and performing UI actions based on eye focus differs fromperforming UI actions based on eye positioning. Eye positioning involvestracking the x and y positions of the user's eyes in order to determinewhich UI element or where the user is looking. Eye positioning does notinvolve tracking the z-depth or distance at which the user is looking orwhat depths are in focus.

Eye focus relies principally on the z-depth or distance of the user'sfocus to perform different UI actions. In some embodiments, the eyefocus tracking is combined with the eye position tracking to performadditional UI actions or to perform additional interactions with a 3Denvironment that do not require or involve hand-generated inputs.

FIG. 1 illustrates an example of performing a UI action based on eyefocus tracking in accordance with some embodiments presented herein. Inthis example, a user wears a mixed reality or augmented reality headsetor device 101. Headset or device 101 includes and/or executes 3D UIcontroller 100.

3D UI controller 100 generates and/or presents (at 102) interactive 3Denvironment 103 in a display of headset of device 101. Interactive 3Denvironment 103 includes a set of UI elements 105-1, 105-2, 105-3, and105-4 (hereinafter collectively referred to as “set of UI elements 105”or individually as “UI element 105”) distributed within a near plane orforeground of interactive 3D environment 103. Set of UI elements 105include windows of different applications or programs, files, mediacontent, and/or different interactive objects.

Set of UI elements 105 partly or wholly obscure the background or farplane of interactive 3D environment 103. In the visualization generatedby mixed reality or augmented reality headset or device 101, thebackground is partly or wholly made up a real-world scene that is infront of the user. Interactive 3D environment 103 presents set of UIelements 105 over the real-world scene. In some embodiments, thebackground of interactive 3D environment 103 may include additional UIelements for other windows or objects that are virtually positionedbehind set of UI elements 105 in the foreground or near plane. In avirtual reality headset or device, the background may be generated fromcaptured images of the real-world scene in front of the user or adigital scene that is partly or wholly obscured by set of UI elements105 in the foreground.

3D UI controller 100 tracks (at 104) the user's eye focus, and retainsset of UI elements 105 in the foreground of interactive 3D environment103 so long as the user's eye focus remains on the foreground and/or atthe depth of one or more of the set of UI elements 105.

A person enters (at 106) in the background of the user's field-of-viewcausing the user's eye focus to shift from the foreground to thebackground. 3D UI controller 100 detects (at 108) the change in theuser's eye focus using one or more sensors in headset or device 101, andadjusts (at 110) interactive 3D environment 103 in response. Forinstance, 3D UI controller 100 detects (at 108) that the foreground ofinteractive 3D environment 103 or one or more planes at a closer firstset of depths is not in focus, and/or that the background of theinteractive 3D environment or one or more planes at a distant second setof depths is in focus based on measurements of the user's eyes (e.g.,the size, shape, reflected light, and/or other properties of the user'spupils).

3D UI controller 100 determines that set of UI elements 105 are in theforeground or positioned at the closer first of depths based onz-coordinate values associated with each UI element of the set of UIelements 105, and adjusts (at 110) set of UI elements 105 in theforeground so as to not obscure the background that the user has shiftedfocus to. As shown in FIG. 1 , 3D UI controller 100 performs an actionthat moves set of UI elements 105 from the user's line-of-sight to theperiphery of the user's sight (e.g., the top, bottom, left, or rightsides). The movements may include shrinking and/or rotating each UIelement 105 on a particular axis so that each UI element 105 ispresented at a size and/or angle that obscures less of interactive 3Denvironment 103. In some embodiments, moving set of UI elements 105includes minimizing each UI element 105 so that they are removed fromthe display. In some embodiments, 3D UI controller 100 adjusts (at 110)the opacity or alpha-channel of each UI element from set of UI elements105 so that each UI element 105 becomes partially or wholly transparentand does not obstruct the user's line-of-sight to the background.

As a result of adjusting (at 110) set of UI elements 105 in theforeground, 3D UI controller 100 opens or clears the user'sline-of-sight to the interactive 3D environment background and/or any UIelements positioned at the distant second set of depths. 3D UIcontroller 100 executes the UI action based on the user changing eyefocus from the foreground or near plane to the background or far planewithout any hand-generated input provided by the user via a handgesture, 3D controller, keyboard, or other input device. In other words,the user simply and intuitively focuses on something in the background,and 3D UI controller 100 automatically adjusts (at 110) interactive 3Denvironment 103 to present the background unobstructed to the user.

The user may engage in conversation or interact with the person in thebackground. To return set of UI elements 105 back to the user'sline-of-sight, the user simply shifts eye focus from the background orthe distant second set of depths to the foreground or closer first setof depths. 3D UI controller 100 detects the eye focus shifting back tothe foreground, and adjusts set of UI elements 105 to their originalpositions in the user's line-of-sight.

FIG. 2 illustrates example components of device 101 for performing theUI adjustments based on tracked changes to the user's eye focus inaccordance with some embodiments presented herein. Device 101 includesdisplay 201, eye focus tracking sensor 203, and 3D UI controller 100that is implemented by one or more hardware processors 205 of device 101or a computing device that is connected to device 101.

In some embodiments, device 101 is a headset, glasses, or other wearabledevice that is positioned before the user's eyes or face. In some otherembodiments, device 101 is a tablet, desktop computer, or othercomputing device that creates a virtual reality, mixed reality, augmentreality, spatial computing, or other 3D experience without being tied tothe user body.

Device 101 may include additional components depending on the usagemodel. For instance, device 101 includes a camera to capture areal-world scene in front of the user and to generate a virtual reality,mixed reality, or augmented reality experience with UI elementspresented in combination with the real-world scene. Device 101 includesmemory, storage, network, and/or other hardware resources for onboardexecution of 3D UI controller 100 or for remote streaming of the UIelements or the 3D UI environment to device 101.

Display 201 presents different UI elements in a virtual reality, mixedreality, augmented reality, spatial computing, or other 3D environment.The UI elements include digital features of the 3D environment thatpresent data or may be interacted with to perform differentfunctionality. For instance, the UI elements may include windows of openor running applications, windows of open images, audio, media, and/orother files, icons for accessing the applications, files, and/or data,icons for invoking or executing different functionality or commands,interactive gaming elements, digital objects or artifacts, and/or otherinteractive or viewable elements that may be overlaid or otherwisepresented at different positions in the 3D environment.

Eye focus tracking sensor 203 is directed towards the user's eyes. Eyefocus tracking sensor 203 tracks eye focus based on one or moremeasurements obtained from one or more of the user's eyes. In someembodiments, eye focus tracking sensor 203 includes an emitter thatgenerates non-visible light for measuring the eye focus. In someembodiments, eye focus tracking sensor 203 includes a receiver thatcaptures images or performs different measurements of the user's eyesfor the eye focus determination.

3D UI controller 100 receives the output from eye focus tracking sensor203, and determines the eye focus or changes to the eye focus based onthe sensor output. For instance, 3D UI controller 100 computes adistance measurement or depth range that the user eye is focusing on, ortracks the user focus transitioning between near and far planes.

3D UI controller 100 includes an interface to the 3D environmentpresented in display 201. In particular, 3D UI controller 100 includesan interface to the window manager of the 3D environment or operatingsystem. 3D UI controller 100 controls the presentation of UI elements inthe 3D environment using the interface and based on tracked changes tothe eye focus. For instance, 3D UI controller 100 issues commands to thewindow manager via the interface to adjust the positioning or opacity ofthe UI elements at different distance or depths in the 3D environment.In some embodiments, 3D UI controller 100 performs other actionsincluding closing, opening, saving, changing the frontmost UI element orUI element that has key-focus (e.g., the UI element that receives userinputs), and/or otherwise changing the UI element selection.

FIG. 3 illustrates eye tracking examples performed by eye focus trackingsensor 203 in accordance with some embodiments presented herein. Eyefocus tracking sensor 203 illuminates (at 302) the pupil of the user eyewith invisible near-infrared or infrared light using a light emitter orlight source. The emitted light does not irritate or interfere with theuser eye or user vision.

Eye focus tracking sensor 203 records and/or measures (at 304) theamount of light reflecting off the pupil to determine the pupil size ornarrowing or expansion of the pupil. From the pupil size or changes tothe pupil size, eye focus tracking sensor 203 and/or 3D UI controller100 determines the eye focus or the depth at which the eye is focused onor whether the user changes focus from a near plane to a far plane orvice versa.

In some embodiments, eye focus tracking sensor 203 measures (at 306) thedistance between the eyes. Specifically, eye focus tracking sensor 203measures (at 306) the center positions of the eyes relative to oneanother as the eyes focus on close and far objects. When a user isfocused on a near plane (e.g., objects or UI elements that are close ora first set of distances from the user), the eyes rotate or turn closertogether such that the angle separating the eyes and the in-focus objectis more acute or smaller. When a user is focused on a far plane (e.g.,objects or UI elements that are far or a second set of distances fromthe user), the eyes rotate or turn away from one another such that theangle separating the eyes and the in-focus object is larger.

In some embodiments, eye focus tracking sensor 203 images or measures(at 308) the surface curvature of the eye lens. For instance, the lensbecomes less concave or curved as the eye focuses on a closer object,and becomes more concave or curved as the eye focuses on a fartherobject.

These eye focus tracking techniques differ from eye positional trackingtechniques that track the x and y positions of the eyes to determinewhere about a two-dimensional plane the user is looking. In contrast,the eye focus tracking techniques implemented by eye focus trackingsensor 203 and used by 3D UI controller 100 are unconcerned with the xand y position where the user is looking, and measure or determine thedepth, plane, or z-coordinate that is in focus.

FIG. 4 presents a process 400 for controlling a UI using eye focus inaccordance with some embodiments. Process 400 is implemented by 3D UIcontroller 100 as part of a system or device that generates a 3Denvironment or 3D UI, and that tracks user eye focus with eye focustracking sensor 203.

Process 400 includes presenting (at 402) a 3D environment with differentUI elements positioned and/or presented at different depths of the 3Denvironment. Presenting (at 402) the 3D environment includes generatingthe UI elements in a digitally created 3D scene or in a real-worldscene. Accordingly, the 3D environment may be a mixed reality, a virtualreality, an augmented reality, a spatial computing, 3D graphical userinterface (“GUI”), operating system, or other environment for selecting,moving, opening, closing, and/or otherwise interacting with thedifferent UI elements.

Process 400 includes tracking (at 404) a z-coordinate position or depthof each UI element in the 3D environment. In some embodiments, each UIelement is defined with a set of x, y, and z coordinates in the 3D spaceof the 3D environment. In some embodiments, the coordinates are definedrelative to a current render position or position of the user in the 3Denvironment. In some embodiments, the UI elements are restricted topositions in the 3D environment foreground or near plane. In some otherembodiments, the UI elements may be distributed at any depth or positionin the 3D environment.

Process 400 includes measuring (at 406) the user eye focus. Differenttechniques may be used to determine the distance of the user focus orwhich depths or planes the user's gaze is focused on or are in focus forthe user eye. For instance, the pupil size, pupillary distance betweenthe center of the eyes, amount of light reflecting from the user eye,and/or other techniques may be used to measure (at 406) and/or determinethe user eye focus.

Process 400 includes determining (at 408) if the user eye focus is inrange of a set of UI elements in the 3D environment foreground or nearplane. The set of UI elements may include the frontmost UI elements inthe 3D environment that are positioned within a near set of z-coordinatepositions, or the z-coordinate position of the UI element withkey-focus.

In response to determining (at 408—Yes) that the user eye focus is inrange of the set of UI elements in the foreground or near plane, process400 includes retaining (at 410) the positioning of the set of UIelements unless the user provides explicit input to select, close, move,or otherwise reposition one or more of the set of UI elements. Forinstance, the user may select and maximize or reposition a particular UIelement of the set of UI elements with a hand gesture, mouse input,trackpad input, 3D controller input, keyboard input, or otherhand-generated input. Alternatively, the user may select multiple UIelements within a selection window, and may issue a command to minimize,close, or reposition the multiple UI elements with a hand gesture, mouseinput, trackpad input, 3D controller input, keyboard input, or otherhand-generated input.

In response to determining (at 408—No) that the user eye focus isfocused on the 3D environment background or far plane (e.g., a far setof z-coordinate positions that are behind or at a greater depth than thenear set of z-coordinate positions), process 400 includes automaticallyperforming (at 412) an adjustment to one or more of the set of UIelements that are in the foreground or near plane. Performing (at 412)the adjustment includes modifying the one or more UI elements to preventthe background or far plane of the 3D environment from being obscured orblocked by the positioning of the one or more UI elements in theforeground or near plane.

In some embodiments, performing (at 412) the adjustment includespresenting UI elements that are in the background or far plane or thatare positioned behind the set of UI elements. In some other embodiments,performing (at 412) the adjustment includes providing an unobstructed orminimally obstructed view of the real-world scene behind the set of UIelements.

In some embodiments, performing (at 412) the adjustment includesincreasing the opacity of the one or more UI elements to make the one ormore UI element transparent and allow the user to see through the one ormore UI elements in the foreground. In doing so, the view of the 3Denvironment background becomes less obstructed through the partially orfully transparent one or more UI elements.

In some embodiments, performing (at 412) the adjustment includerepositioning the one or more UI elements to prevent the one or more UIelements from blocking a view of the 3D environment background or farplane. Repositioning the one or more UI elements may include minimizingthe UI elements, grouping the UI elements to the sides or corners of the3D environment, moving the UI elements to the periphery, staggering theUI elements against one another, and/or tilting or rotating the UIelements about one axis such that the UI elements are presented at anangle that allows the region of the 3D environment behind the UIelements to become visible.

In some embodiments, 3D UI controller 100 performs (at 412) theadjustment to the entire set of UI elements in the foreground or nearplane of the 3D environment. In some other embodiments, 3D UI controller100 performs (at 412) the adjustment to the one or more UI elements fromthe set of UI elements that are at the center of the 3D environment orare positioned in the direction where the user is looking which isdetermined using one or more eye tracking techniques.

FIG. 5 illustrates examples of the different adjustments for exposingthe 3D environment background or far plane in accordance with someembodiments presented herein. Unadjusted 3D environment 501 presentswith the set of UI elements in the foreground or near plane obscuringthe viewing of the 3D environment background or far plane.

First adjustment 503 illustrates the 3D environment with thetransparency of the set of UI elements being increased so that the 3Denvironment background or far plane is visible through the set of UIelements in the foreground or near plane. Second adjustment 505illustrates the set of UI elements being repositioned to the peripheryso that the center of the 3D environment background or far plane isunobstructed. Third adjustment 507 illustrates the set of UI elementsbeing rotated about an axis so as to block less of the background or farplane.

In some embodiments, the eye focus tracking may be combined with eyeposition tracking to implement additional UI controls. FIG. 6illustrates an example of different UI adjustments that are implementedbased on controls derived from different combinations of eye focus andeye position tracking in accordance with some embodiments presentedherein.

For instance, the user has a set of application windows open in the 3Denvironment with a first application window having key-focus andreceiving input from the user. A person walks (at 602) into the room,and the user detects the motion via a region of the 3D environment thatis not obstructed by the set of application windows.

3D UI controller 100 detects (at 604) the user eye focus changing basedoutput from eye focus tracking sensor 203 of device 101. Specifically,3D UI controller 100 detects (at 604) the user eye focus changing fromthe near plane in which the first application window is positioned tothe far plane.

In response to the detected (at 604) eye focus change, 3D UI controller100 moves (at 606) the set of application windows out of the user'sline-of-sight. Moving (at 606) the set of application windows providesthe user with an unobstructed view of the 3D environment far plane, andthe user may engage in conversation with the person that walked into theroom.

The conversation may relate to data from a second application window ofthe set of application windows. The user may look to the repositionedsecond application window. An eye position tracking sensor of device 101generates output that 3D UI controller 100 uses to determine (at 608)that the user is looking at an x and y position that is associated withthe repositioned second application window.

3D UI controller 100 may highlight (at 610) or otherwise identify thatthe second application window has been selected based on the eyeposition tracking. The user may perform an eye tracking gesture (e.g.,blink twice) or may continue to stare at the highlighted secondapplication window for a duration of time. In response to the eyetracking input, 3D UI controller 100 adjusts (at 612) the positioning ofthe second application window next to the person speaking in thebackground. Consequently, 3D UI controller 100 allows the user to pausework on the first application window, direct focus to the person thatthe entered room without obstruction from the set of applicationwindows, engage in conversation with the person, and reposition thesecond application window containing data for the conversation next tothe person engaged in the conversation without any hand- generatedinputs.

In some embodiments, the eye focus controls may be adapted to performother functionality beyond providing an unobstructed view of the 3Denvironment background or far plane. For instance, the eye focuscontrols may be used to close applications, save application state,cycle between different applications, and/or provide data regardingstate in the background.

FIG. 7 illustrates an example of adapting the eye focus controls forswitching between UI elements in accordance with some embodimentspresented herein. The 3D environment presented (at 702) on user device101 includes a set of UI elements with a first UI element that isfrontmost and has key-focus. For instance, the first UI element may be atext editing application for a first document the user is working on.

The user decides to switch from the first UI element to a second UIelement and perform operations using the second UI element. Intraditional UIs, the user has to locate the second UI element in the 3Denvironment, and select the second UI element to transfer key-focus fromthe first UI element to the second UI element. Alternatively, the usermay use an Alt-Tab or other keyboard command to cycle through the set ofUI elements.

In FIG. 7 , 3D UI controller 100 adapts the eye focus controls to allowthe user to cycle between UI elements. For instance, the user changeseye focus off the first UI element or the near plane where the first UIelement is located, and focuses on the background or far plane of the 3Denvironment. 3D UI controller 100 detects (at 704) the change in eyefocus, and activates (at 706) UI element switching functionality inresponse to the change in eye focus and/or in response to no person ormovement being detected in the far plane.

Activating (at 706) the UI element switching functionality includesarranging the set of UI elements in a specified ordering. The specifiedordering may be based on when each UI element was last accessed or theorder in which the UI elements were opened. 3D UI controller 100 beginsto cycle through the set of UI elements by rotating the set of UIelements and bringing a next UI element in the specified ordering to thebackground while the user focus remains on the background. For instance,3D UI controller 100 rotates the set of UI elements every half secondthe user's focus remains on the background. Since the user's focus is onthe background, the user sees the contents of the UI element that ismoved to the background and/or selected. When the user identifies thedesired second UI element in the background they would like to transfercontrol to, the user shifts their focus back to the foreground and 3D UIcontroller 100 detects (at 708) the change in eye focus back to theforeground or near plane.

In response to detecting (at 708) the eye focus changing from thebackground or far plane back to the foreground or near plane after thesecond UI element has been selected, 3D UI controller 100 makes (at 710)the second UI element the frontmost UI element, and transfers key-focusto the second UI element such that subsequent inputs provided by theuser via one or more input devices are provided to the second UI elementinstead of the first UI element. The user may continue switching betweenthe set of UI elements in the same manner by shifting eye focus back tothe background.

The eye focus tracking may also provide controls for video games and/ormultimedia content. For instance, a video game character may be renderedat a far plane or background of a 3D environment. To control and viewthe character, the user's focus shifts to the far plane or background,and the user uses a game controller to provide hand-generated inputsthat move the character in the 3D environment and that cause thecharacter to perform different actions.

Supplemental data about the character or options to change the characterproperties may be invoked by the user shifting eye focus from the farplane or background to the near plane or foreground. 3D UI controller100 detects the user eye focus shifting to the near plane or foreground,and in response, generates an overlay that provides the supplementaldata about the character or options to change the character properties.For instance, the game may generate an overlay in the foreground with aninventory of items that may be used to customize the character or anoverlay that provides health and/or other status about the character.Inputs from the game controller may be diverted to selecting between theoptions in the overlay. Once a selection is made or the user wishes toreturn to the game play, the user need only shift their eye focus backto the far plane or background. 3D UI controller 100 detects the eyefocus returning to the far plane or background, removes the overlay,and/or transfer inputs from the game controller back to controlling thecharacter gameplay.

In this manner, the eye focus controls may be used to quickly accessmenus or the supplemental data without hand-generated input or inputfrom the game controller. For instance, the buttons of the gamecontroller may be assigned to other controls.

In some embodiments, similar eye focus controls may be used to editmultimedia content. For instance, an application may playback videocontent in the far plane or background. When the user wish to perform anedit, the user shifts eye focus to the foreground. 3D UI controller 100detects the change in eye focus, pauses playback of the video content,and presents various tools for editing the video content in theforeground.

FIG. 8 is a diagram of example components of device 800. Device 800 maybe used to implement one or more of the devices or systems describedabove (e.g., 3D UI controller 100, device 101). Device 800 may includebus 810, processor 820, memory 830, input component 840, outputcomponent 850, and communication interface 860. In anotherimplementation, device 800 may include additional, fewer, different, ordifferently arranged components.

Bus 810 may include one or more communication paths that permitcommunication among the components of device 800. Processor 820 mayinclude a processor, microprocessor, or processing logic that mayinterpret and execute instructions. Memory 830 may include any type ofdynamic storage device that may store information and instructions forexecution by processor 820, and/or any type of non-volatile storagedevice that may store information for use by processor 820.

Input component 840 may include a mechanism that permits an operator toinput information to device 800, such as a keyboard, a keypad, a button,a switch, etc. Output component 850 may include a mechanism that outputsinformation to the operator, such as a display, a speaker, one or moreLEDs, etc.

Communication interface 860 may include any transceiver-like mechanismthat enables device 800 to communicate with other devices and/orsystems. For example, communication interface 860 may include anEthernet interface, an optical interface, a coaxial interface, or thelike. Communication interface 860 may include a wireless communicationdevice, such as an infrared (“IR”) receiver, a Bluetooth ® radio, or thelike. The wireless communication device may be coupled to an externaldevice, such as a remote control, a wireless keyboard, a mobiletelephone, etc. In some embodiments, device 800 may include more thanone communication interface 860. For instance, device 800 may include anoptical interface and an Ethernet interface.

Device 800 may perform certain operations relating to one or moreprocesses described above. Device 800 may perform these operations inresponse to processor 820 executing software instructions stored in acomputer-readable medium, such as memory 830. A computer-readable mediummay be defined as a non-transitory memory device. A memory device mayinclude space within a single physical memory device or spread acrossmultiple physical memory devices. The software instructions may be readinto memory 830 from another computer-readable medium or from anotherdevice. The software instructions stored in memory 830 may causeprocessor 820 to perform processes described herein. Alternatively,hardwired circuitry may be used in place of or in combination withsoftware instructions to implement processes described herein. Thus,implementations described herein are not limited to any specificcombination of hardware circuitry and software.

The foregoing description of implementations provides illustration anddescription, but is not intended to be exhaustive or to limit thepossible implementations to the precise form disclosed. Modificationsand variations are possible in light of the above disclosure or may beacquired from practice of the implementations.

The actual software code or specialized control hardware used toimplement an embodiment is not limiting of the embodiment. Thus, theoperation and behavior of the embodiment has been described withoutreference to the specific software code, it being understood thatsoftware and control hardware may be designed based on the descriptionherein.

For example, while series of messages, blocks, and/or signals have beendescribed with regard to some of the above figures, the order of themessages, blocks, and/or signals may be modified in otherimplementations. Further, non-dependent blocks and/or signals may beperformed in parallel. Additionally, while the figures have beendescribed in the context of particular devices performing particularacts, in practice, one or more other devices may perform some or all ofthese acts in lieu of, or in addition to, the above-mentioned devices.

Even though particular combinations of features are recited in theclaims and/or disclosed in the specification, these combinations are notintended to limit the disclosure of the possible implementations. Infact, many of these features may be combined in ways not specificallyrecited in the claims and/or disclosed in the specification. Althougheach dependent claim listed below may directly depend on only one otherclaim, the disclosure of the possible implementations includes eachdependent claim in combination with every other claim in the claim set.

Further, while certain connections or devices are shown, in practice,additional, fewer, or different, connections or devices may be used.Furthermore, while various devices and networks are shown separately, inpractice, the functionality of multiple devices may be performed by asingle device, or the functionality of one device may be performed bymultiple devices. Further, while some devices are shown as communicatingwith a network, some such devices may be incorporated, in whole or inpart, as a part of the network.

To the extent the aforementioned embodiments collect, store or employpersonal information provided by individuals, it should be understoodthat such information shall be used in accordance with all applicablelaws concerning protection of personal information. Additionally, thecollection, storage and use of such information may be subject toconsent of the individual to such activity, for example, throughwell-known “opt-in” or “opt-out” processes as may be appropriate for thesituation and type of information. Storage and use of personalinformation may be in an appropriately secure manner reflective of thetype of information, for example, through various encryption andanonymization techniques for particularly sensitive information.

Some implementations described herein may be described in conjunctionwith thresholds. The term “greater than” (or similar terms), as usedherein to describe a relationship of a value to a threshold, may be usedinterchangeably with the term “greater than or equal to” (or similarterms). Similarly, the term “less than” (or similar terms), as usedherein to describe a relationship of a value to a threshold, may be usedinterchangeably with the term “less than or equal to” (or similarterms). As used herein, “exceeding” a threshold (or similar terms) maybe used interchangeably with “being greater than a threshold,” “beinggreater than or equal to a threshold,” “being less than athreshold,”“being less than or equal to a threshold,” or other similarterms, depending on the context in which the threshold is used.

No element, act, or instruction used in the present application shouldbe construed as critical or essential unless explicitly described assuch. An instance of the use of the term “and,” as used herein, does notnecessarily preclude the interpretation that the phrase “and/or” wasintended in that instance. Similarly, an instance of the use of the term“or,” as used herein, does not necessarily preclude the interpretationthat the phrase “and/or” was intended in that instance. Also, as usedherein, the article “a” is intended to include one or more items, andmay be used interchangeably with the phrase “one or more.” Where onlyone item is intended, the terms “one,” “single,” “only,” or similarlanguage is used. Further, the phrase “based on” is intended to mean“based, at least in part, on” unless explicitly stated otherwise.

What is claimed is:
 1. A method comprising: presenting athree-dimensional (“3D”) environment comprising one or more userinterface (“UP”) elements that are positioned in a foreground of the 3Denvironment and that partially or wholly obscure a background of the 3Denvironment; determining that a user eye focus is focused at a firstdepth corresponding to the foreground of the 3D environment; retainingpositions of the one or more UI elements in the foreground thatpartially or wholly obscure the background in response to determiningthat the user eye focus is focused at the first depth; detecting achange in the user eye focus from the foreground to the background; andproducing an unobstructed view to all or part of the background byadjusting the one or more UI elements in the foreground in response todetecting the change in the user eye focus.
 2. The method of claim 1,wherein detecting the change in the user eye focus comprises:determining that the user eye focus has shifted from the first depth toa second depth corresponding to the background of the 3D environment. 3.The method of claim 1, wherein detecting the change comprises: measuringa change to a pupil with a sensor.
 4. The method of claim 1, whereindetecting the change comprises: measuring a change in an amount of lightbeing reflected off a user's eye with a sensor.
 5. The method of claim1, wherein producing the unobstructed view comprises: increasing atransparency of the one or more UI elements.
 6. The method of claim 1,wherein producing the unobstructed view comprises: moving the one ormore UI elements to a display periphery.
 7. The method of claim 1,wherein producing the unobstructed view comprises: minimizing the one ormore UI elements.
 8. The method of claim 1 further comprising: detectinga second change in the user eye focus from the background to theforeground; and removing said adjusting of the one or more UI elements,wherein said removing comprises returning the one or more UI elements topositions in the foreground prior to said adjusting and to visibleproperties specified for the one or more UI elements prior to saidadjusting.
 9. The method of claim 1 further comprising: tracking a depthat which the one or more UI elements are positioned in the 3Denvironment; determining that the one or more UI elements are positionedin the foreground of the 3D environment based on said tracking of thedepth; and issuing a command that adjusts a positioning or opacity ofthe one or more UI elements in response to determining that the one ormore UI elements are positioned in the foreground and the change in theuser eye focus from the foreground to the background.
 10. A computingsystem comprising: one or more hardware processors configured to:present a three-dimensional (“3D”) environment comprising one or moreuser interface (“UP”) elements that are positioned in a foreground ofthe 3D environment and that partially or wholly obscure a background ofthe 3D environment; determine that a user eye focus is focused at afirst depth corresponding to the foreground of the 3D environment;retain positions of the one or more UI elements in the foreground thatpartially or wholly obscure the background in response to determiningthat the user eye focus is focused at the first depth; detect a changein the user eye focus from the foreground to the background; and producean unobstructed view to all or part of the background by adjusting theone or more UI elements in the foreground in response to detecting thechange in the user eye focus.
 11. The computing system of claim 10further comprising: an eye focus tracking sensor that is directed in adirection of a user's eye, wherein the eye focus tracking sensordetermines the user eye focus based on one or more of a change to apupil of the user's eye or a change in an amount of light beingreflected off the user's eye.
 12. The computing system of claim 11,wherein the computing system is one of a virtual reality, mixed reality,or augmented reality headset.
 13. The computing system of claim 10,wherein detecting the change in the user eye focus comprises:determining that the user eye focus has shifted from the first depth toa second depth corresponding to the background of the 3D environment.14. The computing system of claim 10, wherein producing the unobstructedview comprises: increasing a transparency of the one or more UIelements.
 15. The computing system of claim 10, wherein producing theunobstructed view comprises: moving the one or more UI elements to adisplay periphery.
 16. The computing system of claim 10, whereinproducing the unobstructed view comprises: minimizing the one or more UIelements.
 17. The computing system of claim 10, wherein the one or morehardware processors are further configured to: detect a second change inthe user eye focus from the background to the foreground; and removesaid adjusting of the one or more UI elements, wherein said removingcomprises returning the one or more UI elements to positions in theforeground prior to said adjusting and to visible properties specifiedfor the one or more UI elements prior to said adjusting.
 18. A methodcomprising: presenting a three-dimensional (“3D”) environment comprisingone or more user interface (“UP”) elements that are positioned in aforeground of the 3D environment and that partially or wholly obscure abackground of the 3D environment; detecting a first change in a user eyefocus from the foreground to the background; producing an unobstructedview to all or part of the background by adjusting the one or more UIelements in the foreground in response to detecting the change in theuser eye focus; detecting a second change in the user eye focus from thebackground to the foreground; and removing said adjusting of the one ormore UI elements, wherein said removing comprises returning the one ormore UI elements to positions in the foreground prior to said adjustingand to visible properties specified for the one or more UI elementsprior to said adjusting.
 19. A computing system comprising: one or morehardware processors configured to: present a three-dimensional (“3D”)environment comprising one or more user interface (“UP”) elements thatare positioned in a foreground of the 3D environment and that partiallyor wholly obscure a background of the 3D environment; detect a change ina user eye focus from the foreground to the background; produce anunobstructed view to all or part of the background by adjusting the oneor more UI elements in the foreground in response to detecting thechange in the user eye focus; detect a second change in the user eyefocus from the background to the foreground; and remove said adjustingof the one or more UI elements, wherein said removing comprisesreturning the one or more UI elements to positions in the foregroundprior to said adjusting and to visible properties specified for the oneor more UI elements prior to said adjusting.
 20. A method comprising:presenting a three-dimensional (“3D”) environment comprising one or moreuser interface (“UP”) elements that are positioned in a foreground ofthe 3D environment and that partially or wholly obscure a background ofthe 3D environment; tracking a depth at which the one or more UIelements are positioned in the 3D environment; determining that the oneor more UI elements are positioned in the foreground of the 3Denvironment based on said tracking of the depth; detecting a firstchange in a user eye focus from the foreground to the background;issuing a command that adjusts a positioning or opacity of the one ormore UI elements in response to determining that the one or more UIelements are positioned in the foreground and further in response to thechange in the user eye focus from the foreground to the background; andproducing an unobstructed view to all or part of the background byadjusting the one or more UI elements in the foreground in response toexecuting the command.
 21. A computing system comprising: one or morehardware processors configured to: present a three-dimensional (“3D”)environment comprising one or more user interface (“UP”) elements thatare positioned in a foreground of the 3D environment and that partiallyor wholly obscure a background of the 3D environment; track a depth atwhich the one or more UI elements are positioned in the 3D environment;determine that the one or more UI elements are positioned in theforeground of the 3D environment based on said tracking of the depth;detect a first change in a user eye focus from the foreground to thebackground; issue a command that adjusts a positioning or opacity of theone or more UI elements in response to determining that the one or moreUI elements are positioned in the foreground and further in response tothe change in the user eye focus from the foreground to the background;and produce an unobstructed view to all or part of the background byadjusting the one or more UI elements in the foreground in response toexecuting the command.